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- - - Abstract. In this paper, explicit error bounds are derived in the approximation of rank k projections of 

certain n-dimensional random vectors by standard fc-dimensional Gaussian random vectors. The bounds 
, are given in terms of k, n, and a basis of the fc-dimensional space onto which we project. The random 

■ vectors considered are two generalizations of the case of a vector with independent, identically distributed 

(N ■ components. In the first case, the random vector has components which are independent but need not have 

pH ' the same distribution. The second case deals with finite exchangeable sequences of random variables. 

1. Introduction 

The classical central limit theorem says that, under mild conditions, the random variable Sn = J^i^i -^i 
is approximately Gaussian, for a sequence of n independent, identically distributed random variables Xi and 
• n large. That is, if X is a random vector of M" with i.i.d. components, then the orthogonal projection of 

I X in the direction (1,1,..., 1) is approximately Gaussian. It is natural to ask for what other directions the 

projection of such a random vector is approximately Gaussian. In particular, the Berry-Esseen theorem for 
sums of independent, non- identically distributed random variables implies that for X as above. 



o 



X 



n 

sup|P[(6',X) < t]-^{t)\ < 0.8(E\Xif)y"\e 



> 

' where $ denotes the standard normal distribution function and is any unit vector in M". Thus {0,X) is 

, close to Gaussian as long as X]J=i I^P small. Roughly, this happens as long as there aren't a small number 

of coordinates of 6 controlling the value of {9,X); i.e., the components of 6 are all of similar size. More 
generally, one could ask when higher rank projections of X are close to Gaussian; that is, not only consider 
' the asymptotic normality of individual projections of X, but the asymptotic independence of projections 

, in different directions. The basic result of this paper is the following quantitative bound on the distance 

from a rank k projection of X to a standard Gaussian random vector in a fixed dimension, where distance 
is measured here by comparing the integrals of test functions. In the following theorem, C^(IR'^) denotes 
the space of compactly supported, real-valued functions on M*^ with two continuous derivatives; |||V(7(a;)| ||oo 



I is the maximum length of the gradient of g and |(?|2 = max sup 



dxidxj 



(x) 



The £p norm of a vector 9 G 



is denoted \\9\\p = {Yh=i 



Theorem 1. Let Xi, . . . , Xn be independent, identically distributed random variables with EXi = and 
MXi = 1. Let 9i,...,9k be fixed vectors in K" with 9i = {9},..., 9"), such that {9i,9j) = 5ij. Define a 
random vector Sn G by 

n 

Sn ^ = 91 Xr- 



r=l 



Then for g G C, 



(1) 



lg{Sn) - Eg{Z)\ < ^\\\Vg\\\^^EXt-l (j2 H^^H') 



\k'\gU{E\X,\')(Y^\\9 




where Z is distributed as a standard normal random vector in R*^ . 
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The following example is useful to consider. Suppose the 9i are such that — for each j; i.e., the 6i 
are orthogonal unit vectors in the directions of corners of the hypercube. As long as n is large and a multiple 
of 4, there are more such vectors than we can make use of - see [2]. Then the norm expressions in the bound 
above reduce to 

Thus for directions chosen in this way, projections of rank k are close to Gaussian as long as fc = o (n-'^/®). 
Furthermore, if 9i is random on the sphere, then 



nmi< 



\ 



and 

r=l \2 + 2J 

(See [3| for a straightforward approach to integrating even-degree monomials over the sphere; the proof given 
there extends to odd-degree monomials in the absolute values of coordinates as well.) It follows that for the 
Oi chosen at random (subject to the orthogonality condition), there are absolute constants ci and C2 such 
that for every g e C2(R'=), 

Ee{\EgiSnM)-mZ)\) < ■^[cik^^^\\\Vg\\\oo^fixf~^ + c2k%\2E\Xi\ 

This implies that a typical projection of rank k is close to Gaussian for k = o(n^/^). 

Theorem [T] is generalized below in two directions. In the following version, the Xi are assumed to be 
independent, but need not be identically distributed. 

Theorem 2. Let Xi, . . . , Xn be independent (not necessarily identically distributed) random variables with 
MXi = and EXf = 1 for each i. Let 9i, ... ,9k be fixed vectors in M" with 9i = [9], . . . , 9"), such that 
{9i,9j) — Sij. Define a random vector Sn G R*^ by 

n 

Sn ^ {(^i^^) 91 Xr. 

r=l 

Then for g e C2(R'=), 

\Eg{Sn) - Eg{Z)\ < ^\\\Vg\\\^ I max^EXf - I (j^ 

(2) ^ ^ 

4 ' ^ ^ 



+ maxE|X,|M^ 



3 \l<J<n 

where Z is distributed as a standard normal random vector in . 



ll^.lli , 

i=l I 



Theorem [5] can be generalized further to require the vectors 9i only to be linearly independent. Let Hg(x) 
be the Hessian matrix of g at x, and let 

llll-ffglloplloo = sup 1 1 iJg (a;) 1 1 op 

X 

where ||^||op is the operator norm of the matrix A. Thus || ||i?g||op||oo is the supremum over x of the largest 
eigenvalue (in absolute value) of Hg{x). 

Theorem 3. Let Xi, . . . , Xn be independent (not necessarily identically distributed) random variables with 
EXi = and EXf = 1 for each i. Let 9i,. ..,9k be fixed, linearly independent vectors in R" with 9i — 
(9\, . . . , 0"), such that \\9i\\i = 1 for each i. Let Cij = {9i, 9j) . Define a random vector Sn & R'^ by 

n 

Sn = (^ii -'^) = X! ^i-^"" 
r=l 
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and let Z be a Gaussian random vector with covariance matrix C = (cy)^^^]^. Then for f £ C^(R*''), 
|E/(5„)-E/(Z)| < i%/Afc|l|V/||U./max EX^-l 

Z \ l<i<n \ ^ — ^ 



Hill 



-Xk'{\\\\Hf\U\^) maxE|X,n El""" 



^l<i<r 

where X is the largest eigenvalue of C. 

Theorem [3] follows from Theorem [5] using a fairly straightforward linear algebra argument. 

Theorem [T] can also be generalized in a different direction, by weakening the independence assumption. 

In the following version, the sequence Xi,...,Xn is assumed to be exchangeable, i.e., (Xi,...,X„) = 
. . . , Xo-(„)) for any permutation a, but {Xi, . . . , X„) need not have independent entries. Theorem|3]is 
not a generalization of Theorem [T] in the strictest sense, as it has the additional technical requirement that 

J2r ~ for each i. In what follows, let \g\i = maxi<i<fc 

Theorem 4. Let (Xi, . . . , Xn) be a finite exchangeable sequence of random variables with KXi = and 
E,Xi — 1. Let {9i}^^i be an orthonormal set of vectors in R", such that X]r=i — ^ f'^^ each i. Define the 
random vector Sn in R*^ by 

n 
r=l 

Then there are absolute constants a, b, c such that for any g £ C^(R'^), 

\Eg{Sn)-Eg{Z) \ < ak\g\i ^^EX^X^X^ + y^E(X^^I)(X|~T 

+ b\gU^EXf II^'^IU) + ck^\9\2mi\' ll^^lls) ■ 

In the case that the entries are independent, the first two error terms vanish; one can interpret their 
presence as a requirement that the dependence among the Xi must be weak. 

In the same way as one obtains Theorem [3] from Theorem [2l one can weaken the orthonormality re- 
quirement on the 0i of Theorem 2] to the requirement that they be linearly independent. This yields the 
following. 

Theorem 5. Let {Xi, . . . ,X„) be an exchangeable sequence of random variables withMXi = andlKXi — 1. 
Let {Oi}^^^ be a linearly independent set of vectors in R", such that X]r=i ^ ^ f'^^ each i. Let Cij = {9i, 6j) . 
Define the random vector Sn in M.^ by 

n 

Sn = = E ^i-^r, 

r=l 

and let Z be a Gaussian random vector with covariance matrix C = {cij)ij^i. Then there are absolute 
constants a, b, c such that for any g £ C^(R'''), 

\Eg{Sn)-Eg{Z)\ < akVX\\\Wg\\\oo (^^J\EXiX2X3X4\ + ^J\E(Xf - - 1)|^ 



(5) / k \- / k 

1 , 



+ bVx\\\Vg\\\oo,/EXf ( E II^.IU J + ck'x{\\\\H,\U\o<)E\X,\^ ( ^ I 
where X is the largest eigenvalue of C . 
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2. Proofs 

Proof of Theorem\^ from Theorem\^ Perform the Gram-Schmidt algorithm on the set of vectors {9i}'. since 
the 9i are Hnearly independent, there is an invertible matrix B such that for ?/; := _B,^ 6j, {rji,rij) 
By assumption, 

Cij = {6i,6j) 



\ p q I 



Thus BB'^ ^C. 

Now, let f ■.R'' and define /i : M''^ ^ M by h{x) = f{Bx). Define S], := X^^^i Vl^r- Then 

and so h{Sn) = f{BSn) — f{Sn)- If ^ is a standard Gaussian random vector, then h{Z) = f(BZ) and BZ 
is a Gaussian random vector with covariance matrix BB^ = C. Applying Theorem [2] for this test function 
h to the random vector S',\ yields 

\EfiSn)-^f{BZ) \ = \Eh{Sn)-Eh{Z)\ 



k^\hU (^^max^E|X,|3^ (j2\ml^. 



so it remains to estimate |||Vft.|||oo and \h\2. 

To estimate |||V/i|||oo, first note that if B is viewed as an operator on M.^, then its operator norm is its 
largest singular value. That is, 

\\Bx\\ < VA||x||, 

where A is the largest eigenvalue of C. Since || |V/i|||oo is the Lipschitz constant of h, it follows that || |V/i|||oo < 

Vaiiiv/iiu. 

To estimate \h\2, note that 

=y^BrpBs,J^{Bx) = {B^Hf{Bx)B),p, 

where {Hf)ij — ( a^. ) is the Hessian of /. Now, 

\B^HBUp = I {B^HBe,,ep) \ = \ {HBe,,Bep) \ < \\H\\op\\B\\lp. 
As stated above, ||-B||op < A, the largest eigenvalue of C; this completes the proof. 

□ 

Remarks: 

(1) To obtain a bound in Theorem [3] which doesn't involve the operator norm of the Hessian of /, one 
can estimate the operator norm by the Hilbert-Schmidt norm: 



^ r,s — 1 ^ 
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(2) The proof of Theorem [S] from Theorem 2] is exactly the same as the proof above. 

The proofs of Theorems [2] and [4] are apphcations of the following abstract normal approximation theorem, 
proved in [1_ 

Theorem 6. Let X and X' he two random vectors in R*"' such that L{X) — L{X'), and let Z G M*^ he a 
standard Gaussian random vector. Suppose there is a constant A and random variahles Eij such that 

(1) E[Xl-X,\X] =-XX, 

(2) E [{Xl - X,){X'^ - X,)\X] - 2A% + E,,. 

Then ifge C'^R''), 

\EgiX) - mZ)\ < min | Mi ^ E\E.,\, ^"'J/"'°° e(^ 
(6) I ''^ 

i 

In the contexts in which Theorem [5] is applied below, the pair of vectors X, X' will in fact be constructed 
not only to have the same law but to be exchangeable. 

Proof of Theorem\^ From the random vector Sn with SJ^ = {9i,X)^ make an exchangeable pair of vectors 
(5,1, S'„) by choosing / g {1, . . . ,n} at random, independent of {^i}, and replacing Xj by an independent 
copy Xj. That is, 

Then 



1 " 1 



n 



thus the proportionality condition of Theorem [6] holds with X — 
Next, 



E 



= -j2^mr{Xi- x*,r\{x,}]^,] 
i=i 



n 
1=1 



l + lY.{e[f{xj-i)^ 

n. n ^ — ^ 



n n 



since E^(^f)' = 1- Thus one can take E,, = ^ ELi(^f)'(^l " !)• 
If * J, 

E WJ - SI) {{S'J - Si) |{X,}ILi] = E {6ld][Xi X}f\ 

n 

= -Y^dw,E{{Xr-x:f\{x,Y,=i\ 

r— 1 
1 " 

r— 1 

n 

= -y^s\ff,{xi-x)^ 



n 

r=l 
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where the last hne follows because J2r = 0- Thus ^ i YTr^i - 1) for all i and j. Now, 



k ^ k n 

E4 = iE E^^^r^^^K^^-i)(^^-i) 



i,j=i l,r=l 



Making use of the facts that E(X| — 1) = and Xg and are independent for i ^ r, together with the 
Cauchy-Schwarz inequality yields 



Now, 



and so 



Finally, 



\ ^ 



1, 



-E. 



E E ei9l9^^9^^{X^-l){X^-l) 



\ i,j=ie,r=l 



< 



\ 



k n 

"^1 E E 0i9l9p^^{Xf-l){X?-l) 

£,r— 1 



\ 



,i=i i=i 



2(ft^\2^Xj - 1)2 



< 



maxjEX4-l) ^ ^(^^f)2(0^^)2 

i,j=i 1=1 



Y.{9lf{9'^^ < 



E 



\ £=1 \ £=1 



E(^i 



1 1 4 1 1 1 1 4 ' 



E4-^^(e«) /ii;^^- 



ms'j~si,\' ^-Y,\9i\^nx,-x^ 



*|3 



^-Ei 



nmxji' 



< 



Ml max E|X,f , 

71 l<i<n 



where the second line follows from the triangle inequality and the fact that X* has the same distribution 
as Xj . The statement of the theorem is now an immediate consequence of Theorem 121 □ 



Proof of Theorem^ Starting from Sn, make an exchangeable pair of random vectors as follows. Choose a 
pair of indices I ^ J at random from {1, . . . , 7i}. Let r ~ (IJ), the permutation on n letters that transposes 
/ and J, and let 

X' — (XtI^x)-, ■ ■ ■ , -'^T(n))- 

Then 

iS'nY = {(^ii X') ^ Sl^ + - 9^ Xj + 9i Xi - 9^ Xj, 

that is, 

(7) {S'J~Si, = {9i-9'!){Xj^Xj). 
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Let denote summing over distinct indices. Then 



n — 1 



where the second line follows as J^r ^ Thus the proportionality condition of Theorem [6] holds with 



A = 



ri-l ■ 



The next step is to compute and bound the error terms Eij. First, consider i — j. From ([7|), 

E [iis'j^ - s':,r\x] = J2\9: - etnx., - x^? 



nin - 1) ^ 



1 



n{n — 1) 



n{n — 1) 



4 2 

+ 



n — 1 n(n — 1) 



tnx,~Xrr 

r,s r,s 

-iY^9^^9tx^ + Aj20lXretx, 

r,s r.s 

Y^x?+nY^i9lrx'^-2(Y^i9^,rx^ (y^^s 

r \ r / \ s 

2(Ym'x)i (YXs]+2{Slf 



i \2 



2(5;) 



The error En can thus be taken to be 
2 



Yixi i)+nYm\xi - 1) - 2 (Ym'x\ (yx, 

_ r r \ r / \ s 



2{S, 



•i \2 



n{n — 1) 

To bound ElE'iil, first apply the triangle inequality and treat each of the four terms above separately. First, 



E 



e(^{x?-i){x^-i)^ 



< ^n\E{Xf -l)\ + ^n{n- 1)|E [{Xf - - 1) 



Next, 



E 



Ym'{x^-i) 



< 



\ 

< /|E [xf i]| + iiE [{x! - i){xi - i)]i E(^r)'(^i)' 

y r r,s 

< m\l^\EXf - 1| + [{Xf - - 1)] |. 
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Note also that the normahzation is such that E(S',\)'^ = 1, thus only the second-last term remains to be 
estimated. As before, start by applying the Cauchy-Schwarz inequality: 



E 



Now, breaking up the sum by the equality structure of the indices and using the exchangeability gives 



/e {ef?{ei?XkXtXrXs 



< 



(n^\E[XiX2X3Xi] I + n^\mt\E[XfX2X3] \ + n\miE[XfXi] + 2nm\i\E[XfX2] 

1 

+ \\e,\\iE[Xt] + n\E[XfX2X3] I + 2\E[XfX2] \ + 2n\E[XfX2X3] \ + 2E[XfXi]^ ' 



< n 



EX1X2X3X4I + \Ml^/\EXfX2X3 



2\EXfX2\ + \EXfXl + J3\EX^X2X3 



e.WlJEXf + J2\EX'fX2\ + j2EXfX, 



By Holder's inequality and the exchangeability of the sequence, 

jEXfXzl < EX^, \EX^X^\ < EX^, and \EXfX2X3\ < EX^. 

Also, 

^< 11^.11^ <1 
since 116*^112 = 1. Thus there are constants c, c' such that 



E J2 (efyiOD^XkXeXrXs < cn^\EXiX2X3X4\+ c'n\\9^\\l^EXf. 

y k,e,r,s 

All together, this shows that there are constants Ci, C2, C3 such that 



(8) 



E\Eu\ < -J\EXiX2X3Xi\ + -J\E{X^ - 1) (X| - 1) | + ^\\e,\\lJEXf. 



Next, consider Eij for i ^ j. From (O, 

^[{{sLr - SI) {is:,y - si)\{x,Yu] = ^[{ol - 0:^) {ej - ej) [Xj - x,f\ {x,} 

^ ' k,e 

Expanding this expression and making use of the facts that {6i, 9j) =0 and J^r ^1 ~ Tlr ^j^^ gives that 
the right-hand side is equal to 



n{n — 1) 



n Y 0^0'; Xi - 2 (y ^'^l^fc) (E + ^^nSi 
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As in the case of En, to estimate E|i?y |, apply the triangle inequality to the expression above and estimate 
each term separately. First, 



E 



< 



EXf, 



where the second line follows from exchangeability and the fact that {Oi,0j) — 0. By the normalization, 
|ES',\S'^| < 1, and it remains to estimate the middle term. As before, this is done by applying the Cauchy- 
Schwarz inequality, breaking up the sum by equality structure of the indices, and using exchangeability and 
the orthonormality conditions on the 9i to simplify the result. This process yields 



E 



re^jXrX, 



< 



n^\EXiX2X3X4,\ + n^\EX(X2X3 \ + 2n EATf ATa + nEAf A| + EA^^ 



< 



4cnJEXf, 



for some constant c. It follows that there is another constant a such that for all i ^ j, 



mj\<-\\9M\d,\u^^xt. 



It now follows that 



/j. / 

i E 1 ^ ck^\EX^X2X3Xi \ + c'A:^|E(A2-1)(A|-1)| + c" ^EXf | E I 

— 1 \ i 

1—1 I ■ 1 3 

To complete the application of Theorem[6l it remains to estimate E (S'^)* — S"^ . By ([7]), 



s„ 



E 



H){x.,~xi)\ 



l—Y^Em-6l){X,~Xr)t 



n{n — 1) 



^ 8E|Ai|3 ^ 

^ r.s 



< 



nE(K 



QS I \ 3 



n{n — 1) 

8E|Ail3 
n(n — 1) 

n(n — 1) L 

Here the third line follows from the L3 triangle inequality and exchangeability, and the last line by expanding 
the cube and using the normalization condition on 9i. Note that, by Holder's inequality and the fact that 
116*^11 = 1 for each i, 

and 



> 



1 



thus the second term above can be absorbed into the first with a change in constant. It follows that 

k 



WE\{S'J^-S:f <cE\X,\'J2\\(^. 



A 



J|I3- 
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This accounts for the remaining error term in Theorem [3) 

□ 
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